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1. INTRODUCTION 

University website reflects the university's academic activities. Thus, the more qualified a university, 
the more qualified its website. Some researchers discuss the variety of college sites based on its performance. 
However, there is lack one to another [1]-[3]. They examine the characteristics used for the assessment, then 
do the website ranking. The most ratings used to represented quality are Webometrics Ranking of World 
Universities, uniRank™ (formerly 4 International Colleges & Universities or 4ICU.org), and eduroute. 
Webometrics ranking consists of four criteria such as visibility, size, rich files and scholar [4]. The 
uniRank™ ranking methodology used five indicators from four different web intelligence (Moz domain 
authority, Alexa Global Rank, SimilarWeb global rank and Majestic) [5]. Eduroute ranking methodology 
consists of four criteria, there are volume, links quantity, quality of links and content, and online scientific 
Information. The disadvantage of webometrics methods is only suitable for universities with a high 
commitment to publication. While uniRank™ and eduroute just focus on the popularity of the websites. 

Based on some university ranking shortcomings, We tried to look at website quality based on 
different perceptions. One of the research on website quality shows that website quality evaluation criteria 
focus on usability and accessibility [1]. Accessibility is a level of scope of users to achieve certain goals [6], 
[7]. Meanwhile, usability covers the simplicity in learning, effectiveness, and pleasure for the users [6], 
[8]-[11]. The factors affected accessibility and usability represented by load time, stickiness, and backlink. 
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The experiments performed by Dominic and Hanim (2013) used Fuzzy AHP with Extent Analysis (EA) 
methods on the quality of university websites in Malaysia based on usability criteria [1]. The deficiency of 
most FAHP applications using the EA method proposed by Chang (1996) is considered invalid and the 
weight obtained by this method does not represent the relative importance of the criterion or decision criteria 
[12]-[16]. To address the deficiencies of the FAHP method with EA, then Mikhailov (2004) proposed the 
Fuzzy Preference Programming (FPP) method to derived the crisp weight of the pairwise matrix comparison 
[17]. Nevertheless, there are some shortcomings of FPP methods proposed by Mikhailov (2004). The lack of 
the FPP method are (1) Negative membership degree makes no sense, (2) FPP model produces multiple 
optimal solutions when there exists strong inconsistency among the fuzzy judgments, (3) The priority vectors 
derived by using the upper or lower triangular elements of a fuzzy pairwise comparison matrix are not the 
same, even significantly different [18]. 

The non linear based Logarithmic Fuzzy Preference Programming (LFPP) is the improved FPP that 
aims to resolve the negative membership degree result and multiple optimal solutions when there exists 
significant inconsistency among the fuzzy judgments [18]. This method uses logarithmic elements of natural 
numbers that can produce a single solution in determining the value of importance weights. This research has 
been integrating LFPP method to rank the quality of University websites. By using this method, it is expected 
that quality ranking of University website can be made better. This paper is organized as follows. Section 2 
discusses literature study; Section 3 describes the result and discussion, and Section 4 presents the 
conclusions and suggests some areas for further investigation. 


2. LITERATURE STUDY 
2.1. University website rank 

Some studies create a method to measure the quality of university websites, such as webometric, 
uniRank™ and eduroute. Table 1 summarizes the university website quality ranking methodology. 


Table 1. Methodology 


Criteria Webometrics Unirank Eduroute 

First Release 2004 2005 2011 

URL Web http://www. webometrics.info http://www.4icu.org/ http://www.eduroute.info/ 

Univ's analyzed 16000 13000 1000 

Univ's ranked 5000+ 200 500 

Size Web size (20%) Volume (20%) 

Research output Rich Files (15%) Online Scientific information 

(10%) 
Google Scholar (15%) 

Impact (Link) Visibility (50%) Majestic Reffering Domains, Links quantity (30%), 
Majestic Trust Flow, Quality of links and content 
Alexa Global Rank, (20%) 
Similar web Global Rank, 


Moz domain authority 


Isidro and the research group from Cybermetrics Lab, Spain, initiate university web ranking based 
on four criteria called webometrics. Since 2004, every six months, they released the rank based on web 
presence and impact. Webometrics analyzes 16.000 universities from various place in the world and ranks 
over 5.000 universities. Webometrics uses google, google scholar, Majestic SEO and ahrefs website tools to 
retrieve the data [4]. In 2005, the uniRank™ University Ranking from Australia launched a relative 
popularity ranking of world Universities and Colleges based on the popularity of their websites regarding 
traffic, trust and quality link popularity [19]. The uniRank position supports the international students and 
teachers to understand the university popularity in a foreign country. Eduroute released the university web 
ranking in 2011 but it did not continue until now. Eduroute focuses on studying and evaluating school 
websites based on volume, links quantity, quality of links and content, and online scientific information [20]. 

The uniRank™ and eduroute use the popularity of websites to rank. The uniRank™ and eduroute 
lead to their method of having full dependence on search engine algorithms. Unfortunately, we do not know 
exactly how the search engines work. Search engine developers hide their algorithms in indexing or crawling 
websites [21]. Webometrics studies the building and using information resources, web structures and 
technologies with bibliometric and infometric approaches. Webometrics does not aim to evaluate web design, 
usability or popularity of contents against the number of visitors [22]. Meanwhile, uniRank™ also has one 
main weakness in which its judgment is developed based solely on the popularity and usage of the website 
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only and does not see the content of the site. The disadvantages of eduroute rating are the unclear method of 
ranking calculation they have and the tools used for data collection. Based on the problems above, it is 
necessary to conduct further study in order to investigate the quality factor of university websites. 


2.2. Website quality study 

There are a lot of discussions about the factor quality of websites. Certain factor, such as 
performance, is important to evaluate [1], [5], [23]. A research on 71 Bangladeshi college websites reveals 
that the important criteria of web presence are the number of web page, simple web impact factor, self-link 
web impact factor and external link web impact factor [24]. Another study shows that based on web 
developer or administrator and user assessment, the websites performance factor represented by accessibility 
and usability [1]. 

The accessibility of a website can be represented by its response time. There are three responses of 
time limit; the first is 0.1 seconds limit that make the users think that the system react immediately. The 
second limit is 1.0 seconds that leads the usesrs thought stay for a while and interrupted. The ultimate limit is 
10 seconds that keeps the users' attention focused on the dialogue. Amerson says that the average server 
response time must be less than 0.5 sec [1], [23], [25]. 

Studies of website usability have shown that slow page rendering is typically caused by not big 
images. But also server delay (because of complex data processing) or overly fancy page widgets. The 
number of a component such as a widget, images, video and sounds that embedded to the websites are also 
the significant factor to be investigated [26]. 

The other criterion of site quality is its loading time. The quality standard of the web page loading 
time must be less than 30 seconds with average server response time must less than 0.5 seconds [27]. 
Loading time is time median required to load a web page on the browser of the users concretely. 

Stickiness is a capability to ensure that each user has been stuck on a page of the website in the 
period of time. A pleasurable site will attract the users to revisit later emerging a positive impact that is traffic 
impact. A backlink is measured from the number of links referring to the main page. It shows the credibility 
of a website for being correlated to the prestige and popularity of a website [1], [5], [23]. Thus, this research 
test three criteria of accessibility and usability, namely stickiness, backlink, and web page loading time. 


2.3. Analytical hierarchy process (AHP) 

AHP is a decision support model developed by Thomas L. Saaty. This decision support model would 
describe multi-criteria problems into particular hierarchy. The first level of the hierarchy is the goal, which is 
followed by the level, criteria, sub-criteria, and so on down to the last stage of the alternative. With certain 
structure, a complex problem can be broken down into groups that are then organized into a hierarchical form 
so that the problem will seem more structured and systematic [28]. The AHP weakness lies in the use of 
unproportionate evaluation scales and is unable to handle the uncertainty of pairwise comparison processes 
[29]. Decision makers are usually involved in making the right choice in fixed judgment. They prefer to use 
interval sense to overcome the comparison process to be more accurate. The AHP method used a crisp 
number to represent human judgment. Along with the increased complexity of the problem, the researchers 
combine the concept of fuzzy logic with AHP which is known as Fuzzy Analytical Hierarchy Process 
(FAHP) [30], [31]-[36]. The number used is no longer the crisp number but the fuzzy number. 


2.4. Fuzzy number 
Chang [37] defined a fuzzy number A on R to be a triangular fuzzy number if its membership 
function 444 (x): R —> [0.1] is equal to 


(m-l °? (1) 


(x) = (u—m) > 


0, otherwise 


Where 1<m<u, l and u stand for the lower and upper value of the support of A respectively, 
and m for the modal value. The triangular fuzzy number (TFN) can be denoted by (/,m,u). The 
membership function of TEN can be seen in Figure 1 [14], [15], [38]. 
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Figure 1. A Triangular Fuzzy Number of A 


Consider two triangular fuzzy numbers A, and A, F A, = (L sM, u) , and A, = (L, m,,u,) then 
A ®A, =(L,m,u)®(L,m,u,)=(l +1,,m,+m,,u, +u,) (2) 


A, 8A, =(L,,m,u,)@(L,,m,,u,)=(1, xl,,m, xm,,u, xu, ) l; > 0,m, >0,u,; > 0 (3) 


A — A, =(1,,m,,u,)—(,,m,,u,) =, —1,,m, —m,,u, =U) (4) 
A, +A, =(1,,m,u,)+(L,,m,,u,)=(1,/l,,m,/m, ,u,/u, ),l, >0,m, >0,u, >0 (5) 
Av = (4, m14) = (1/u, 1/m,,1/1,), 1; >0,m, > 0,u; >0 (6) 


We used the TEN to constructed fuzzy evaluation matrix on AHP method. If the strong importance 
of element j over element iholds, then the pairwise comparison scale can be represented by the fuzzy 


number a; = (z, 1). 


2.5. Fuzzy linguistic variables 

Linguistic variables can be defined as a variable that represents different aspects of human language. 
It can be displayed by fuzzy numbers that suitable to constitute a degree of subjective judgment. Meanwhile, 
assessment weights are expressed in linguistic variables with values; “Very unimportant”, “Unimportant”, 
“Fair”, “Important”, “Very important” or “Very Poor”, “Poor”, “Fair”, “Good”, and “Very Good” [39]-[41]. 
Table 2 shows the nine basic linguistic terms that are used in this research. Each membership function (scale 
of fuzzy number) is defined by three parameters of the symmetric triangular fuzzy number, the lower point, 
middle point, and upper point of the range over which the function is defined. 


Table 2. Membership Function Linguistic Scale [23] 


Convert from AHP scale to Fuzzy number Linguistic expressions 
1=(1,1,1) Equal 
2 = (1,2,3) Equal-moderate 
3 = (2,3,4) Moderate 
4 = (3,4,5) Moderate-fairly strong 
5 = (4,5,6) Fairly strong 
6 = (5,6,7) Fairly strong-very strong 
7 = (6,7,8) Very strong 
8 = (7,8,9) Very strong-absolute 
9 = (8,9,9) Absolute 
2,4,6,8 Values between two adjacent assessments 
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2.6. Logarithmic fuzzy preference programming 

Wang explained that LFPP method is a refinement of the FPP process. In FPP method, a few cases 
resulted in the final value of the negative course this makes the solution is expected to be less valid [18]. 
LFPP method involves a logarithmic function original to correct deficiencies FPP method. 

We used a fuzzy numbers in the assessment in questionnaires by users who have knowledge of the 
issues to be tested, in this case, university website quality ranking. Then a fuzzy pairwise comparison matrix 
can be expressed as 


1 (Lz: Ma,lha) t+ (Lig My Uy) 
À = Õe z Caranta) ! - ses EA (7) 
CLM Ur) (hMer Ups) 0 1 


where 


l; = Yu; oN, =l/m; My =I; and Us l; i mij = uij for all i, j = 1,2,...,k, j #1 (8) 


k 
To find a crisp priority vector W = (w s Wo seess Wg yr >0 with > W, = 1 for the fuzzy pairwise 
i=l 
comparison matrix [18]. The approximate equation uses natural logarithmic numbers for the improvement of 
a paired matrix of fuzzy (7). 


Ina, z~ (in Z, ln m, ln u), i, j =1,2,...,k. (9) 
The membership function of triangular fuzzy judgment can accordingly be defined as 

In\w, /w, J-InL, 

Lear Og P In Mj» 

(i ( j= ln m; —ln l; 

aae = Inu, -In(w,/w,) 

a OAIN daw) Inm,, 

Inu, —Inm, (10) 


where 4l; (In(w, / W; )) is the membership degree of In(w, / w,) belonging to the approximate triangular 
fuzzy judgment In a, = In@;,m,, 


A=min |, (In(w,/w,)}i=12,..k Lj =i+1..kf 


u) . To maximize the minimum membership degree, can be found by 


the crisp priority vector . The resultan model can be 


constructed as 


Maximize Å 
AIn(w,. /w, })> Ai =1,2,....k—1;j =i +1,...,k, 
Subject to ml (w/o; ) ' P (11) 
w, 2 0,1 =1,2....,k, 


or as 


Maximize 1— A 
In w, -ln w, —An(m, Jl; 
Subject to -ln w, +In Wj —In(u; Jm) —In Ui» 
w; 2 0,i =1,2,...k. 


)2Ind,,é=1,2,..,k-Ly =i4,....k 
i=1,2,...,.k -—Lj=itl...k, (12) 
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k 
Ye 
The normalization constraint ‘=! is not included int the above two equivalent models. This is 


because the models will become computationally complicated if the normalization constraint is included. 


2 p= 
W; 21 for all l= 1,2,...k 


Before normalization, without loss of generality, can be assummed that such that 


Inw20 . i=12..k In w, > 0i =1,2,...,k) 


or 
negative value for A produced because there are no weights that can meet all the fuzzy judgments in A 
Inw,—Inw,—Aln(m, /l,)2In1, 


. The nonnegative assumption for is not essential. A 


within their support intervals. That is to say, not all the inequalities 


—Inw,+Inw, —Aln\u, /m, )=—Inu, 
: _ l Y: / y ) ü can hold at the same time. To avoid À from taking a negative 


value, Wang, 2011 [18] introduced nonnegative deviation variables Ôj and Ni for i= 1,2,...,k —1, and 


j=i+l,..., k such that they meet the following inequalities : 


jf =12,..,k-Ly =i+..k, 


ij? 


mM.. 
Inw,-Inw, snl Mo, zt 


i 


u.: 


-ln w, +lnw, aaf en> Inu;,i =1,2,....k -Lj =i+1,...k. 
m 


ij 


The LFPP method formulated as 


J=(1-ay +PY Sl +17) 
Minimize ial Jarl (13) 
x,-x,-Aln(n, /1,)+6, 2 InJ,,i=12,...k-Lj =i+ Lk 
Subject to |—x, +x, —An(u, /m, )+ N; 2 -ln upi =1,2,..,k -l;j=i+1,...,k, 
Ax, 2 0,i=1,2,...,k, 


pn; 20,1 =1,2,...k -Ly =i+1,...k, 


Let x, (i =1,2,...,k) be the optimal solution to model. The normalized priorities for fuzzy pairwise 


comparison matrix A = (d;;),,, can be obtained as 


ema Gi i=L2,.. k ae 
exp(x;) 


j=l 


3. RESULTS AND DISCUSSION 

This study is an experimental research involving website developer to evaluate websites. The 
alternative websites consists of five Indonesian university websites and their url address were : UGM 
(http://ugm.ac.id/), ITB (https://www.itb.ac.id/), UI (http://www.ui.ac.id/), UB (https://ub.ac.id/), and IPB 
(http://ipb.ac.id/). The data of the website quality was taken from 20 to 30 March 2017. The criteria of the 
website quality is taken though the web diagnostic tools http://alexa.com for page loading time, stickiness 
and backlink. The decided hierarchy for selecting the best university based on the quality of site can be seen 
in Figure 2. 
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Selecting the best 
university websites 


Vv Vv Vv 
C; (Stickiness) Cv (Backlink) C; (Load time) 
AA | Vv | Vv 
UGM ITB UI UB IPB 


Figure 2. Hierarchical structure for university websites selection problem 


Table 3 shows the pair wise comparison matrix for ranking university websites. The goal of this study to 
select the best university websites. There are three selection criteria, C, (Stickiness), C) (Backlink), and C, 
(Load time). 


Table 3. Fuzzy Pairwise Comparison Matrix of three selection Criteria 


Criteri 

Triterla C C; C, 

C G,1,) (5, 6,7) C1, 2, 3) 
1 

C (1/7, 1/6, 1/5) (1, 1,1) (1/4, 1/3, 1/2) 
2 

C (1/3, 1/2, 1) (2, 3, 4) a, 1,1) 
3 


2 3 
Minimize J =(1— A)’ + Py: Sle + n2) 
i=l j=i+l 
X, — x, —Aln(6/5) + 6,, = In(5), 
=x, +x, —AIn(7/6) + N, 2 —In(7), 


Subject to xX, — x, —Aln(2)+6,,; 2 In(1), 
=x, +x, — Aln (3/2) +7,, >—In(3), 


xX, — x, —AIn(4/3) + ô»; = In(1/4), 
=x, +x, —Aln(3/2)+7,, 2 —-In(1/2), 


By solving model (13) for this fuzzy pairwise comparison matrix, we obtain the optimal solution as 
A = 0.999, 6,y=-2.564*10, 1i =-2.859*10", x, = 1.564, 
Ô= 3.81010", 13=1.121*10", x, = -0.227, 


Ô= 1.663*10", 133= -9.421*10"!, xX; = 0.871. 
Based on which, we have normalized LFPP priorities as 


w; = exp(1.564) K(exp(1.564) + exp(—0.227) + exp (0.871)) = 0.6, 
w, =exp(—0.227) (exp(1.564) + exp(—0.227) + exp (0.871)) =0.1, 
w, = exp (0.871) (exp (1.564) + exp (0.227) + exp(0.871)) = 0.3, 
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From the equation above, we produce 7’ = 0.999 ~ 1, it means that this set of priority match fuzzy 
pairwise comparison matrix of three selection criteria in Table 3 has high consistency. Using the same way, 
the weighted assessment criteria pairwise comparisons between alternatives can calculated. Table 4 shows 
the data of quality criteria in five university websites. Measurement units for the criterion of load time and 
stickiness were second, and backlink was number. 


Table 4. Original Data 


Criteria UGM ITB UI UB IPB 
Cı — Stickiness (second) 256 268 346 272 268 
C2 — Backlink (number) 8653 4758 6830 4415 4954 
C; — Load time (second) 1.745 1.455 1.507 1.535 1.779 


Table 5 shows the results of the normalized data measurement using Simple Additive Weighting 
(SAW) for five alternative websites in higher institution. SAW methods are used to generate the greatest 
value to be selected as the best alternative. This method requires the process of normalizing the matrix to a 
scale comparable to all current alternative ratings. The normalization results ranged from 0 to 1. 


Table 5. Normalized Data 


Criteria UGM ITB UI UB IPB Weight 
Cı 0.833 1 0.965 0.947 0.817 0.6 
C2 0.739 0.774 1 0.786 0.774 0.1 
C3 1 0.549 0.789 0.510 0.572 0.3 


Table 6 shows multiplication of values by weights generates the aggregation of the local priorities and rank; 
the result represents that UI as the first rank and IPB as the fifth rank. The alternatives ranked as follow: 
UI>UGM>ITB>UB>IPB. 


Table 6. Aggregation of the local priorities 


University websites 


No Criteria UGM ITB UI UB IPB 

T G 0.4998 0.6000 0.5790 0.5682 0.4902 
2 Co 0.0739 0.0774 0.1000 0.0786 0.0774 
3 C3 0.3000 0.1647 0.2367 0.1530 0.1716 
Sum 0.8737 0.8421 0.9157 0.7998 0.7392 
Rank 2 3 1 4 5 


4. CONCLUSION 

The rank and performance of a university can be viewed from its academic activities on the internet 
that are reflected in the university website. Therefore, university website quality is important to show the 
credibility of a college. Based on the obtained data, it can be arranged that the site quality is ranked due to its 
stickiness, backlinks and load time by the LFPP method. The equation of model achieve high consistency of 
the set priority matching to fuzzy pairwise comparison matrix of three selection criteria. The results of this 
study show that stickiness is the most important factor that affects website's quality. For further study, criteria 
and alternatives to test whether the method works well can be added. 
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